Skip to content
This repository has been archived by the owner on Oct 2, 2023. It is now read-only.

Fast pulls and pushes #71

Merged
merged 5 commits into from
Jul 6, 2017
Merged

Fast pulls and pushes #71

merged 5 commits into from
Jul 6, 2017

Conversation

mattmoor
Copy link
Contributor

This is a work in progress on top of #70 building tighter integration between docker_pull, docker_push and the new intermediate form of docker_build in the diffbase PR.

Right now this just consists of a docker_import rule, which will be useful for folks checking in base images to a mono-repo to still get performant builds. It will also be an implementation detail of docker_pull in a subsequent change.

For now, I'm simply opening this PR to run CI over my branch :)

@mattmoor
Copy link
Contributor Author

@damienmg @dlorenc I'll do the remainder of the work in this branch, but I'm going to checkpoint pieces of it. If you want a more complete picture of where things are headed, watch this PR. :)

@mattmoor
Copy link
Contributor Author

The remaining changes need some changes in google/containerregistry, which are out for internal review. I'll post the docker_pull and docker_push changes here once I can update the google/containerregistry version to pull in a new release with the changes we need.

@mattmoor
Copy link
Contributor Author

The google/containerregistry changes are in, I'll cut a release once Travis comes back happy.

@mattmoor
Copy link
Contributor Author

Ok, this PR should contain the full set of changes now.

@mattmoor
Copy link
Contributor Author

Note to self, update docs to drop :image.tar everywhere.

@mattmoor mattmoor force-pushed the fast-pull branch 4 times, most recently from 14d0c74 to 4e896e3 Compare June 28, 2017 23:12
@mattmoor mattmoor mentioned this pull request Jun 28, 2017
Closed
@jmillikin-stripe
Copy link

This is a work in progress on top of #70 building tighter integration between docker_pull, docker_push and the new intermediate form of docker_build in the diffbase PR.

For users that want to build and transport Docker images without running a Linux-only daemon as root, the Bazel docker rules and pusher.par are currently the only game in town. Rather than treating this new image layout as an opaque implementation detail of docker_push, consider naming it and documenting its structure.

Then you'd have the additional advantage over docker save and docker push in that people could figure out how to integrate other tooling into the pipeline without having to crack open the Docker CLI source code.

@mattmoor
Copy link
Contributor Author

@jmillikin-stripe Not exactly the intent of the intermediate format, but I definitely acknowledge the broader sentiment. I will open an issue to track documenting the format on-disk, as well as the intermediate representation in this repo.

@mattmoor mattmoor mentioned this pull request Jun 29, 2017
Closed
@mattmoor
This implements fast docker_{push,pull}
f246dc8 
This commit pulls in the latest release of `google/containerregistry`, which includes a new fast-path for puller/pusher to use, which is consistent with our new intermediate format.

This commit adapts `docker_pull` to consume the new puller and stitch together the image's constituent elements using `docker_import`.

This commit also changes `docker_push` to consume the new options on the new pusher to publish images without linking the full `docker save` tarball, which can be quite expensive for large images.
@mattmoor
Fix the layer ordering from docker_pull
7fc241a
@mattmoor
Remove the reversed layer ordering after picking up the fix in our di…
6d56cda 
…ffbase.
@mattmoor
Remove another unused utility.
5ef844c
@mattmoor
Update the documentation to drop :image.tar everywhere
5ee8f1b
@mattmoor mattmoor changed the title [WIP] Fast pulls and pushes Fast pulls and pushes Jun 30, 2017
@mattmoor
Copy link
Contributor Author

@damienmg @dlorenc This should just be a handful of changes now and should be RFAL.

@mattmoor
Copy link
Contributor Author

mattmoor commented Jul 5, 2017

@dlorenc Anything to add?

@mattmoor mattmoor merged commit 2256361 into bazelbuild:master Jul 6, 2017
@ixdy ixdy mentioned this pull request Aug 24, 2017
Merged
k8s-github-robot pushed a commit to kubernetes/kubernetes that referenced this pull request Aug 26, 2017
Merge pull request #51297 from ixdy/bazel-fast-docker_pull
2616513 
Automatic merge from submit-queue (batch tested with PRs 50889, 51347, 50582, 51297, 51264)

bazel: use fast docker_pull

**What this PR does / why we need it**: takes advantage of bazelbuild/rules_docker#71.

Faster builds = yay.

**Release note**:

```release-note
NONE
```

/assign @Q-Lee @spxtr @mikedanese
sudarshang pushed a commit to sudarshang/rules_docker that referenced this pull request Sep 26, 2019
@jin
Move android-ndk example to android/ (bazelbuild#71)
81540a9
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@damienmg damienmg damienmg approved these changes

@dlorenc dlorenc dlorenc approved these changes

Assignees
No one assigned
Labels
cla: yes
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@mattmoor @jmillikin-stripe @dlorenc @damienmg @googlebot